Systems-based accident analysis methods: A comparison of Accimap, HFACS, and STAMP

Three accident causation models, each with their own associated approach to accident analysis, currently dominate the human factors literature. Although the models are in general agreement that accidents represent a complex, systems phenomenon, the subsequent analysis methods prescribed are very different. This paper presents a case study-based comparison of the three methods: Accimap, HFACS and STAMP. Each was used independently by separate analysts to analyse the recent Mangatepopo gorge tragedy in which six students and their teacher drowned while participating in a led gorge walking activity. The outputs were then compared and contrasted, revealing significant differences across the three methods. These differences are discussed in detail, and the implications for accident analysis are articulated. In conclusion, a modified version of the Accimap method, incorporating domain specific taxonomies of failure modes, is recommended for future accident analysis efforts. Keywords: Accidents, accident analysis, led outdoor activities, Accimap, HFACS, STAMP. Accepted version of manuscript (2011) 2 Introduction Accidents and accident causation remain key themes within Human Factors research efforts worldwide. It is now generally accepted that accidents represent a complex systemsphenomenon in that causal factors reside at all levels of complex sociotechnical systems, and interact across them (e.g. Leveson, 2004; Rasmussen, 1997; Reason, 1990). It is also acknowledged that our understanding of accidents remains incomplete and that accidents will continue to occur within complex sociotechnical systems (Hollnagel, 2004). This arises not as a function of poor research; rather, it reflects the evolving, probabilistic complexity inherent in how accidents unfold. The methods that researchers, practitioners, and accident investigators use to analyse or investigate accidents are, therefore, critical to aid our understanding of the underlying causes as well as indicating where system safety may be improved. Three accident causation models currently dominate the Human Factors literature: Rasmussen’s (1997) risk management framework (e.g. Cassano-Piche et al, 2009; Jenkins et al, 2010; Johnson & De Almeida, 2008; Salmon et al, 2010; Svedung and Rasmussen, 2002); Reason’s (1990) omnipresent Swiss Cheese model (e.g. Lawton and Ward, 2005); and Leveson’s (2004) Systems Theoretic Accident Modelling and Processes model (STAMP e.g. Ferjencik, 2011; Ouyang et al, 2010). Each engenders its own distinct approach for analysing accidents. Accimap, a generic approach used to identify and link contributory failures across six sociotechnical system levels, accompanies Rasmussen’s risk management framework. The Human Factors Analysis and Classification System (HFACS; Wiegmann and Shappell, 2003), a taxonomy-based aviation accident analysis approach, was inspired by Reason’s Swiss Cheese model. Finally, the STAMP model uses control theory and systems dynamics methods to describe the systemic control failures involved in accidents. Although all three are Accepted version of manuscript (2011) 3 underpinned generally by a systems approach, there are significant differences in terms of theoretical underpinning, the methodological approach adopted, and the outputs produced. Despite this, there is little in the way of guidelines to support the selection of one over the other for accident analysis purposes, and methodology selection is more likely to be based on theoretical preference than anything else. The aim of this paper is to compare and contrast the three methods when used for accident analysis purposes. In doing so an analysis of a recent high profile incident in the led outdoor activity domain, the Mangatepopo gorge incident, is presented. Accident causation and analysis in the led outdoor activity domain There is an acknowledged risk of both severe and frequent injury associated with active pursuits, especially those participated in for sport, active recreation or leisure (e.g. Finch et al., 2007). One popular form of active pursuit is led outdoor activity (defined as facilitated or instructed activities within outdoor education and recreation settings that have a learning goal associated with them, including activities such as school and scout camping, hiking, harness sports, marine aquatic sports and wheel sports; Salmon et al., 2010). Within Australia, injurycausing accidents are currently recognised by the led outdoor activity industry as a significant problem (Salmon et al., 2010). Whilst exact injury rates remain unknown (largely due to the paucity of exposure data), recent high profile fatal incidents, such as the drowning of a 12 year old student during a college camp (Levy, 2010), highlight the industry’s need to understand the causal factors involved and develop appropriate prevention strategies. Recent research, however, indicates that the industry’s understanding of accidents is limited, and that the surveillance systems required to enhance it do not exist (e.g. Salmon et al., 2010). Accepted version of manuscript (2011) 4 Systems-based accident analysis methods have been applied across the other safety critical domains to identify causal factors and inform appropriate system reform and accident countermeasure development. The application of these methods in the led outdoor activity domain has to date been sparse; however, recent evidence suggests that they are likely to be useful as part of an overall accident and injury surveillance and prevention system (Salmon et al., 2010). The present analysis, therefore, involved not only comparing the three methods generally, but also investigating which of the three accident analysis method is most suited for future accident analysis efforts in this domain. Systems-based accident models and methods In this section an overview of the three accident analysis methods, and their theoretical underpinning, is given. Rasmussen’s risk management framework and Accimap Rasmussen’s risk management framework (Rasmussen, 1997) describes the various system levels (e.g. government, regulators, company, company management, staff, and work) involved in production and safety management and considers safety an emergent property arising from the interactions between actors at each of these levels. According to Rasmussen each systemic level is involved in safety management via the control of hazardous processes through laws, rules, and instructions. For systems to function safely decisions made at high levels should promulgate down and be reflected in the decisions and actions occurring at lower levels of the system. Conversely, information at the lower levels (e.g. staff, work, equipment) regarding the system’s status needs to transfer up the hierarchy to inform the decisions and actions occurring at the higher levels (Cassano-Piche et al., 2009). Without this Accepted version of manuscript (2011) 5 so called ‘vertical integration’, systems can lose control of the processes that they are designed to control (Cassano-Piche et al., 2009). According to Rasmussen (1997), accidents are typically ‘waiting for release’; the stage being set by the routine work practices of various actors working within the system. Normal variation in behaviour then serves to release accidents. Rasmussen (1997) outlined the Accimap method, which is used to graphically represent the system wide failures, decisions and actions involved in accidents. Accimap analyses typically focus on failures across the following six organisational levels: government policy and budgeting; regulatory bodies and associations; local area government planning & budgeting (including company management, technical and operational management; physical processes and actor activities; and equipment and surroundings. Notably, Accimap is a generic approach and does not use taxonomies of failures across the different levels considered. Rasmussen’s risk management framework and Accimap method are presented in Figure 1. Figure 1. Rasmussen’s risk management framework and Accimap method. Government Regulators, Associations